Optimizing Return-On-Investment (ROI) for Empirical Software Engineering Studies Working Group Results

Return-on-investment (ROI) is a concept from the financial world. In the dynamic view, ROI describes the periodically recurring profits (returns) from fixed financial capital (investment). In the static view, ROI describes the one-time income or saving (return) realized as a consequence of a one-time expenditure (investment). In this case, if the return does not occur within a short time, later parts of the return may be discounted for interest. For our purposes, we will use the static view and ignore discounting. We will call the return benefit and the investment cost. The notion of ROI can be generalized to any domain (in particular engineering) in which both cost and benefit can be quantified on a rational scale and both scales use the same units. Most commonly, the scale is either money or time. ROI can be used for retrospective analysis of the performance of an investment (controlling) or as an aid for decision-making about potential investments (planning). In the former case, both cost and benefit may be known and the ROI is a single fixed number. Values greater than 1 indicate successful investments, values smaller than 1 indicate failed ones. In the latter case, however, there is usually some uncertainty about the cost and almost always significant uncertainty about the benefit; both expected cost and expected benefit are best described by probability distributions and hence the ROI is also a distribution. In many cases, the uncertainty can be factored into a base case plus a number of identifiable risks, each of which increase cost or reduce benefit.